The present invention relates to the processing of image data representing a matte, key or control signal.
Many compositing techniques are known in which a first image is combined with a second image selectively at positions throughout the image. The technique is known in cinematographic post production in which silhouettes or mattes are used to blank out portions of the image. Similar techniques have been used in broadcast video applications in which a control signal or a key signal is used to selectively derive images from a first video source or from a second image source. Such key signals may also be represented as images or, during the transmission of video data, control signals may be distributed in parallel with image signals in which the control channel is often referred to as the alpha channel.
In cinematographic film it is possible to produce hard edged mattes given the very high spatial definition of the media. However, when images are transmitted as video signals, at broadcast definition or at a higher definition, it is necessary to quantise the spatial images, thereby significantly reducing the availability of spatial definition.
It has been known for some time that the reduced spatial definition of a video image may be compensated by increased depth such that sub-pixel boundaries may be perceived by the eye by averaging colours at the actual position of the pixels. Similarly, control signals or key signal may be produced which provide for soft blending between the value representing a hard-key-on and a value representing a hard-key-off. Typically, such signals are recorded to a depth of eight bits with a similar eight bits being allocated for the red green and blue colour components.
In a process of chroma-keying, soft values for keys may be produced at image edges where pixels will be quantised with colour components being partially derived from a foreground image and partially derived from the background key colour. Thus, it is possible to derive soft-edge keys automatically which may then be used to smoothly blend a foreground image against a new background image. Furthermore, characteristics of the blending edge may be referred to as the softness of the edge and the key values may be modified in order to enhance the properties of softness.
Soft keys can also be used to key transparent and translucent objects against new backgrounds. Soft keys allow the new background colour to be partially mixed with the new foreground and the blue components present within the original foreground image are suppressed by a process of colour suppression; as is well known in the chroma-keying art. However, many translucent and transparent objects are not uniformly transparent and may achieve their transparency by being made up from very small regions which are either very opaque or very transparent. Thus, when used to produce a soft key signal in a chroma-keying process, it is likely that the transparent region will be perceived as noise and will not achieve the required effect.
A system in which each pixel is represented by three colour components defining a position within colour-space is disclosed in International patent publication WO 98/11510. A base colour is identified and a distance in colour-space between an input colour and said base colour is calculated. The control value, which may be used as a basis for chroma suppress or a chroma-key, is produced in response to the calculated distance.
The system disclosed in this publication provides improved procedures for producing a chroma-key, thereby achieving improved chroma-keying results. However, a problem with the system is that it is difficult for users to control operational parameters so as to make optimum use of the processing facilities available.
A system for processing image data taking the form of a plurality of image pixels forms the subject matter for the applicant""s co-pending British patent application (DL P407-GB). A three-dimensional representation of colour-space is displayed and the colour of pixels within an image is analysed. Thereafter, the analysed pixels are displayed at colour related positions within the displayed colour-space.
An advantage of this improved approach is that a user is provided with a graphic visualisation of colour-space regions, thereby allowing colours defined in colour-space to be graphically identified as forming part of a tolerance region or a softness region within colour-space.
A problem associated with defining softness regions is that some textures within an image, such as translucent textures, may produce adverse affects. The development of a chroma-key involves non-linear processes in order to extract the key signal whereupon it is possible for the texture, perceived as noise, to become amplified in parts of the image where intermediate values are generated. This results in clearly visible processing artefacts which could severely restrict the application of the process when textures of this type are present.
According to a first aspect of the present invention, there is provided a method of processing image data representing a matte, key or control signal, comprising steps of identifying a transitional portion of said matte in which undesirable transitions occur in a form displayed as noise artefacts, comprising steps of adjusting the position of a softness region in colour-space so as to reduce the relative amplitudes of variations within said identified portion.
Preferably, the matte is generated by a process of chroma-keying. Preferably, relative amplitudes are reduced by modifying a softness region so as to reduce the presence of high frequency variations. The softness region may be perturbed iteratively, with frequency distributions being measured for each perturbation and a selection of a preferred perturbation being made on the basis of said measurements. The frequency values may be determined by performing a Fourier transform against the softness values.
In a preferred embodiment, a softness region is an ellipsoid and the position of the softness region is modified by transforming the shape of the ellipsoid. A stretching operation may be performed against the ellipsoid and the ellipsoid may be determined by defining a convex hull around a plurality of pixel colours defined in colour-space.
According to a second aspect of the present invention, there is provided image data processing apparatus configured to process image data representing a matte, key or control signal, comprising identifying means for identifying a transitional portion of said matte in which undesirable transitions occur in a form displayed as noise artefacts; and adjusting means configured to adjust the position of a softness region in colour-space so as to reduce the relative amplitudes of variations within said identified portion.
In a preferred embodiment, display means are provided for displaying an ellipsoid to an operator in colour-space and the shape of the displayed ellipsoid may be modified in response to manual operation of a manually operable device.